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The binary symmetric channel 
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Shannon's noisy channel coding theorem 
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For any channel: 
Reliable (virtually error-free) communication is possible 
atrates up to C 
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Emma Woodh*use, hands*me, clever* and rich,*with a 
comfortab*e home an* happy di*position,*seemed to*unite som* 
of the b*st bless*ngs of e*istence;*and had *ived nea*ly 
twenty *ne year* in the*world with very*little *o distr*ss 
or vex*her. *he was*the yo*ngest *f the *wo dau*hters *f a 
most *ffect*onate* indu*gent *ather* and *ad, i* cons*quenc* 
of h*r si*ter'* mar*iage* bee* mis*ress*of h*s ho*se f*om a 
ver* ea*ly *eri*d. *er *oth*r h*d d*ed *oo *ong*ago*for*her 
to*ha*e *or* t*an*an*in*is*in*t *em*mb*an*e *f *er*ca*es*es* 
a*d*h*r*p*a*e*h*d*b*e* *u*p*i*d*b* *n*e*c*l*e*t*w*m*n*a* 
gkkekkekk,kkhkkh*x *k]*knkkikklkkskkrkkokkakkokkekki* 
AkkkokkknkkASkÄKokkkYKKÄSKÄKAÄKKSKEKAKKAIKKKOKKAN KAA 
Μκκκκοκκκκρλκκκ ας κκκκ]κκκκακκκκσκακκηκκκκΕκκκκακκκκρκκκάκνκκκ 


Emma Woodhouse, handsome, clever, and rich, witha 
comfortable home and happy disposition, seemed to unite some 
of the best blessings of existence; and had lived nearly 
twenty one years in the world with very little to distress 
or vex her. She was the youngest of the two daughtersofa 
most affectionate, indulgent father; and had, in conseguence 
of her sister's marriage, been mistress of his house froma 
very early period. Her mother had died too long ago for her 
to have more than an indistinct remembrance of her caresses; 
and her place had been supplied by an excellent woman as 
governess, who had fallen little short of a mother in 
affection. Sixteen years had Miss Taylor been in Mr 
Woodhouse's family, less as a governess than a friend, very 
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twenty *ne year* in the*world with very*little Χο distr*ss 
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ver* ea*ly *eri*d. *er *oth*r h*d d*ed *oo *ong*ago*for*her 
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A simple redundant source - a bent coin 


000000000000000001001001000000000000000000100001000101000000 
000101000000000000000000000100000000010000000000000100000110 
000010000000000000001000000000001000010000010000000000000000 
000010000001000000000001000000000000000000000000010000000000 
000000010001000000100000010010000000000001000000001000000000 
000100100001010100000011000010000000000001000000000000000000 
000000000000000000000000010100000010000000000000100010100001 
100000000010000000000000000000000000000000000000000000000000 
010000000000000100000000000000000100001000000000110110000000 
101000101000000000000000000000000000100010100000100001000000 
000000000000100000100000100001001000000001010001000000000000 
000000000000000000000000011000000001010000000000100100010101 
000110010101000000000000100000000000000000100000001010000110 
100000001000000010110000000010000000010010000010000000000000 
100000000000000100000100000000000000100000000000000000000010 
000010001000000100000000000000100000000001000001000000000000 
110100000100100000000000000000001000000000000000001000000000 
100000000000000000000001000000010010001000000001100000011100 
000000000001000000000000000000000001000000000000001000000000 
000000000010000000000010100000100000000000010000000000001000 
101000100100010000000000000000000000000000000000001010000000 
010010000000000000000000000001000001100000010000000001000000 
000000001000100000000000010000000000000000000000000000001000 
000000000001000000010000000000000000000000000000000000000000 
000001000000000100100000000100000000100001000000100000000000 
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The weighing problem 
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You are given 12 balls, all equal in weight except for one 
that is either heavier or lighter. 
Design a strategy to determine 
which is the odd ball 
and whether it is heavier or lighter, 
in as few uses of the balance as possible. 
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The Bent Coin Lottery 


Lottery tickets available 


A coin with p; = 0.1 will be tossed N = 1000 times. 
The outcome is x = 2122 ...ZN. 
e.8., x = 000001001000100...00010 


You can buy any of the 2" possible tickets for £1 each, 
before the coin-tossing. 


Tf you own ticket x, you win £1,000, 000, 000. 


Q To have a 99% chance of winning, 
at lowest possible cost, 
which tickets would you buy? 


e Andhow many tickets is that? 


Express your answer in the form 207), 


Please think about this guestion before lecture 3 


0000000000.....00000 
0000000000.....00001 


0010000001.....01000 


1111111111.....11101 
1111111111.....11110 
1111111111.....11111 


Homework 


(Exercise 1.9, page 14) 


Design an error-correcting code for the binary symmetric channel 
(or another noisy channel, you choose); invent a decoding algorithm 
for it, and estimate its probability of error. Be creative! 


Other recommended exercises: 1.3, 1.5-7, 1.11; 


Reading: Chapters 1, 2, 4. 
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